Guidelines for TPC members
IEEE
INFOCOM 2012
March 25 - March 30, 2012, Orlando , FL , USA
Questions? Please email TPC Chairs at
infocom2012.chairs@gmail.com
Disclaimer: This
document and the process described are subject to change without notice!
Summary of Action Items for TPC Members
- Accept or Decline the TPC
Invitation (within 1 week of receipt).
- Plan to attend INFOCOM
2012 TPC Informational/Planning Meeting on April 13, 2011 - during INFOCOM
2011 (Highly Recommended).
- Select TPC topics of
interest and update Conflicts of Interest by July 15, 2011 (Required)
- Claim papers by
August 5, 11:59PM PDT.
- Expect to be notified of
all papers assigned to you by August 22, 2011.
- Complete TPC Regular
Reviews for papers by October 1, 2011 (Required)
- Lead/Participate in
online discussion on your assigned papers (October 2-16, 2011)
(Required)
- Write Lead TPC Discussion
Summary/Review by October 17, 2011 (Required)
- Attend TPC Meeting
(Highly Recommended) on November 12, 2011 in
As an IEEE INFOCOM 2012
TPC member, your reviews and participation in the TPC are crucial in
determining the ultimate quality of the conference. Here are some guidelines to
help clarify the TPC work. A simple way to think about the TPC duties is to
think reciprocally about what you expect as an author
of a paper from a peer in the community: a good, substantive review, a decision
that is well-founded on the consensus of the reviews and good feedback. This is
what we expect from TPC members. The element of randomness in the process cannot
be eliminated, but we want to minimize it while biasing it towards merit and
providing good feedback to authors.
Table of
Contents
- Updating
your topics of interest and conflicts of interest information
- Claiming
papers
- Regular
Review vs. Lead TPC Review/Summary
- Accepting
and Delegating Reviews - HOWTO
- Participating
in Online Discussions
- High
Review Quality
- Coherence
between numerical scores and review content
- Review
Delegation guidelines
- TPC
On-line discussion phase
- Role
of Area TPC Chairs
- TPC
Meeting
- Registering
for TPC Meeting - HOWTO
- Guidelines
for TPC Meeting Attendees
Updating your topics of
interest and conflicts of interest information
You are requested to
update your topics of interest for INFOCOM 2012 and your conflicts of interest
information on EDAS after accepting the invitation to serve on the TPC by
following the instructions below. You can update your topics of interest by logging into http://edas.info/ using your login email address
and your EDAS password. Then click on "My TPCs" tab.
You will see a list of conferences in which you serve as a TPC member. In the
row for "INFOCOM 2012", please click on the cell in the "Interests (edit)"
column. The link is usually set as "undefined", by default. In the subsequent
screen, you will see that you are listed as "neutral" to all the topics for the
conference. Please select at least eight (8) topics and click on the
corresponding menu buttons in the "of interest" column. Similarly, please select
as many topics which are not "of interest" to you. Please be sure to click on
"Change topics of interest" button at the bottom of the page to update your
topics of interest. In the list of topics for INFOCOM 2012, please note that
the suffix SIM refers to both Simulation and Experimental work; MOD refers to
Analytical modeling, Performance evaluation and Theory. It is critical that
you update your topics of interest before the review assignment, in order to
make sure that you have the best chance to receive papers which fall in your
topics of interest.
You can update your conflicts of
interest (COI) information by logging into http://edas.info/ using your login email address
and your EDAS password. Then click on "My Profile" tab. On the first line,
please click on the link listed as "conflicts of interest". In the subsequent
screen, you will see a list of individuals with whom you have identified
Conflicts-of-Interest. Please note that in EDAS, "Your initial list of
conflicts-of-interest consists of individuals having the same email domain, the
same name or those that have co-authored EDAS-registered papers with you. You
can provide either the name of the organization, the name of an individual, or
an email domain, or several of these." Please be sure to identify all
individuals with whom you have conflicts of interest, paying particular
attention to potential INFOCOM 2012 authors and TPC members. We greatly
appreciate your efforts in updating this information in EDAS so that we may
avoid unnecessary changes to review assignments later, a time-consuming
process.
Please read the INFOCOM 2012 Definition
of Conflict of Interest for determining the set of individuals you need to list
on EDAS provided below. People/institutions that you are in "conflict of
interest" consist of the following:
- Same institution within 4
years (anywhere in your organization or parent company counts); note that this
includes your previous or future institution if you have recently changed job,
or are in the process of changing job.
- Co-author (or pending) a
publication within 4 years, in any technical area, including patents, tech.
reports, etc.
- Active or pending grant,
proposal, or application of any sort together
- Ph.D, post-doctoral advisor/advisee at any time,
current or past
- Close personal friend (or
enemy!)
- Any other non-technical
bias
Claiming
papers
You can claim (request)
the papers that you would like to review for INFOCOM 2012 by logging into EDAS (http://edas.info) using your login email
address and your EDAS password. Then click on "My TPCs" tab. You will see a list of conferences in which you
serve as a TPC member. In the row for "INFOCOM 2012", please click on the cell
in the "Requested (claim)" column. In the subsequent screen, you will see a list
of papers marked as one of the following: Want to review, Can review, Review if
needed and Cannot review. Please select at least 75
papers and click on the corresponding menu buttons in the "Want to review," "Can
review" and "Review if needed" columns. Similarly, please select as many papers
which you feel you "Cannot review." Please be sure to click on "Change" button
at the bottom of the page to update your list of claimed (requested) papers for
INFOCOM 2012. Please be sure to list at least 75 papers in the Want to
review, Can review and Review if needed columns to provide us with enough
flexibility to assign three independent reviewers to each paper.
Frequently asked
questions
·
I am not able to view the list of abstracts to
claim from? - Detailed Instructions
Please log into EDAS (http://edas.info) using your login email
address and your EDAS password. If you have forgotten your EDAS password, you will need to
follow instructions at http://edas.info to request the system to email you your
password. If you have already logged in to EDAS, you can Claim Papers
by clicking on the "My TPCs" tab at the top, then
going to the row marked "INFOCOM 2012" and clicking on the number (usually 0) in
the corresponding cell in the column marked "Requested (claim)." If the list of
papers (abstracts) that match your interest is large, you will receive a message
such as: "There are *NUM* papers in the list for *your
EDAS ID* - restrict topics." To modify (or select) topics "Of Interest" and
restrict the list of papers that are shown to you, please follow the directions
above. Please make sure that you mark as many topics as possible under "no
interest" and click on "Change topics of interest." Then please try the process
of Claiming papers again. According to the EDAS webpage on selecting topics: "Papers
containing the topics labeled 'of interest' will be included, those containing
topics 'not of interest' will be excluded even if they contain topics of
interest. Papers labeled with the neutral topics will be included if they
also contain topics that are of interest." Please be sure to list at least 75
papers in the Want to Review, Can Review and Review if needed columns (total)
and at least 30 papers in the Want to Review column.
·
I am not able to see the My TPCs tab - Detailed Instructions
Please click on the "My
profile" tab and then click on the "My TPCs" tab.
·
I have too many papers (> 100) claimed in the
categories: Want to Review + Can Review + Review if needed. How do I reduce this
number?
You need to move more
papers to the other category: Cannot Review. This is done by clicking on
the Claim link and then marking a different category (column) for the papers in
the Can Review category, since these constitute the bulk of your selections
typically. Another option for you is to modify your set of topics to mark more
of the topics as "no interest" and try the claims process with fewer abstracts.
Reminder: We would like you to claim at least 75 papers as the sum of the three
categories: Want to Review, Can Review, and Review if
needed. Please also claim at least 30 papers in the Want to Review
category.
Regular Review vs. Lead
TPC Review/Summary
- Each TPC member will be
the TPC lead for about 5 papers and will be a regular reviewer for about 14-16
papers (including the ones for which he/she is the TPC lead reviewer).
- For papers for which you
are the TPC lead, in your EDAS screen (after you login, select "My TPCs" and INFOCOM 2012 from the list of conferences), you
will see both a link to "tpc" as well as "review"
under the "Type" column.
- For papers for which you
are a regular reviewer, you will only see the "review" link under the "Type"
column. First, you need to provide a review for each of the papers in your
list by using the "review" link and filling out the review form by the Regular
Review deadline. You may delegate some regular reviews as described below (see
Delegation).
- After that there will be
an online discussion phase during which you can discuss with other
reviewers/TPC leads each of the papers that has been assigned to you for
review. The messages posted during the online discussion will not be made
available to the authors. Please see below
for instructions on how to participate in the online discussion.
- Once the discussion phase
is over, as the TPC lead you will need to provide a summary of the discussions
in the form of a TPC lead (meta) review. For this,
please click on the "tpc" link under the "Type"
column for your TPC lead papers and fill out the TPC summary form and choose
one of four categories for each paper
- Accept
- Discuss
- Needs Additional Review
and
- Reject.
- Accept
- This TPC lead (meta) review (TPC summary) needs to be completed by the TPC
Summary deadline. The TPC summary will not be made available to the authors.
Accepting and
Delegating Reviews - HOWTO
If you are a TPC member
with primary review responsibilities, you will receive by the deadline, your
review assignments with upto 5 papers for which you
are the TPC lead reviewer and upto 14-16 papers for
which you are the regular reviewer. Please note that you will be assigned a
regular review for each of the papers for which you are a TPC lead reviewer. So
the actual number of papers that you will deal with is only around 14-16.
- How do I accept all of
my review assignments?
When you log
in to your EDAS page (http://edas.info) for INFOCOM 2012 (and click on My
Reviews tab), you will be shown a list of all the papers you are handling for
this conference. Please click on the link at the top of the table which states
"You can also accept and decline reviews in a batch." You will be
presented with a list of all your review assignments with the status Accepted
column marked for each paper. If you have ensured that you do not have a
conflict of interest with any of the authors for the papers in this list, please
go ahead and click on the "Change" button at the bottom of the page.
- How do I accept my
review assignments one by one?
When you log
in to your EDAS page (http://edas.info) for INFOCOM 2012 (and click on My
Reviews tab), you will be shown a list of all the papers you are handling for
this conference.
To accept
your TPC lead (meta) review assignments: In the second
column (marked Type of review), please click on the link marked "tpc" and then click on the "Accept review" button. Please do
this for each of the papers for which you are the TPC lead (meta) reviewer. Please note that you may not delegate your
TPC lead (meta) reviews and you may not decline your
TPC review assignments unless you have a conflict of interest with the authors.
To accept
your regular review assignments: In the second column (marked Type of
review), please click on the link marked "review" and
then click on the "Accept review" button. Please do this for each of the papers
for which you are the regular reviewer. Please note that you may not delegate
your regular reviews to graduate students and you may only delegate a maximum of
one-third (up to 5) of your regular reviews to qualified colleagues and you may
not decline your regular review assignments unless you have a conflict of
interest with the authors.
We request
that you ACCEPT all of your review assignments within a week after receiving the
notification emails and no later than Sep. 1, 2011.
- How do I decline my
review assignments?
Please follow
the instructions above to view the review assignments and instead of clicking on
"Accept review" button, please click on the "Decline review" button. Please
decline reviews ONLY when there is a conflict of interest with the authors of
the paper.
- How do I delegate my
regular review to a qualified colleague?
First, please
read the INFOCOM Delegation guidelines below.
From your INFOCOM 2012 TPC member page, please do the following:
- Click on the paper
number
- Click on the pencil
icon for editing your Review
- Click on Delegate the
review
- Please enter the last
name or EDAS ID of the reviewer. There may be many IDs for an individual and
so please select the active one in consultation with the potential reviewer.
- You will then be able
to edit the Review invitation email sent out by EDAS. Please add content
at the top to let the potential reviewer know that the invitation is coming
from you and not the INFOCOM 2012 TPC Co-chairs (default).
- You will receive a copy
of the completed review by email when your delegate has submitted it.
- Delegation must be
done within 2 weeks of papers assigned to the TPC member.
- The TPC member must
also follow up with their delegates to finish their reviews, and oversee the
quality. Please inform the delegate that they are also co-responsible with
you for actively participating in the online discussion phase.
Participating in
Online Discussions - HOWTO
To post an online
discussion message for a paper that you are reviewing, please do the following
(after the Regular Review deadline).
- Click on the Paper number
(after clicking on My reviews tab upon EDAS login).
- You should be able to
view the reviews and at the bottom of the page, there will be a Discussion
section.
- Click on the Discuss
bubble icon
.
You will see a text box in the subsequent screen. Please type in your comments
about the paper. The discussion messages will be emailed to all the reviewers
and Lead TPC member for that paper. You will also be able to view the
discussion messages online.
- For guidelines on the
online discussion phase, please see below.
As lead TPC member of a
paper, you will need to take an active role in the discussion regarding that
paper. You may ask reviewers to clarify their remarks, provide additional
justification for their comments etc. Once you have completed the discussion
phase, you (as Lead TPC member) will need to summarize the discussion in your
Lead TPC Review/Summary.
High-Quality
Review
High-quality reviews is by far the most important contribution of a TPC
member.
- Basic Quantity/Quality:
Reviews of length shorter than 100 words will generate an automated EDAS
feedback to you requesting more input.
- Kindly do not
cut-and-paste the same information in different categories when this happens:
please enter feedback that will be useful to authors.
- In particular, if any of
the papers are reviewed with the help of students, we expect the TPC member to
provide oversight and clarity and personally enter the reviews into the EDAS
system.
- In the TPC meeting, a
review having a score of 5 with a short unsubstantiated review will be
discounted, and the other dissenting reviews may override the higher score if
they are more substantive.
So, if you
want your reviews to matter in TPC discussions, please enter substantive,
high-quality reviews.
Coherence between
numerical scores and review content
INFOCOM 2012 will adopt
a numerical score range of 1 to 5 for reviews.
The TPC lead assigned to a given paper will have to check if the numerical
scores are well justified by the written reviews (and also catch short/low
quality reviews), and follow up with reviewers using EDAS. Since every TPC
member is a lead for some papers, kindly use this opportunity to check the
coherence of the scores/reviews you give to papers.
For example, if there is a score of 5,
there has to be clear enumeration of reasons why this paper deserves that
rating. (Remember that the scores are logarithmic!) After you have reviewed all
papers, and see the distribution of scores you have given, please see if there
is any bias in your scoring and if it is indeed justified by the quality of the
papers in your set.
Similarly, a rating of 2 or 1 may lead
to a paper not making it to the TPC meeting: so it is the reviewer's
responsibility to give more feedback for authors. Please note that irrespective
of your scores, if a paper has reviews that are not strong, the scores may not
matter.
Delegation
guidelines
The following guidelines
apply to delegation of reviews. In principle, we strongly discourage delegation
unless there is a compelling need such as a very qualified reviewer for a given
paper.
- Lead TPC papers
(typically 5): NO DELEGATION. We expect the TPC member to read the paper and
write the review and actively participate in the post-review on-line
discussion phase. Please note that you may not delegate the regular review for
your Lead TPC papers.
- One-third of regular
reviews (about 5 out of 14-16): Delegate to qualified researcher via EDAS, but
*NOT* graduate students; Delegation must be done within 2 weeks of papers
(September 1, 2011) assigned to the TPC member. The TPC member must also
follow up with their delegates to finish their reviews, and oversee the
quality. Please inform the delegate that they are also co-responsible with you
for actively participating in the online discussion phase.
TPC On-line
discussion phase: Guidelines
- Each TPC member is a TPC
lead for at most 5 papers. In this online discussion phase, the TPC-lead
should try to resolve conflicting reviews and to come up with a unanimous
recommendation for these papers. The TPC-lead is also in charge of writing a
short review (known as the TPC-lead or meta- review), which summarizes the
discussion and makes a recommendation (Accept, Needs additional review,
Discuss at TPC meeting or Reject).
- TPC Leads: Please start
the discussion as soon as two reviews are available instead of waiting for all
three reviews to be submitted. The other reviewers will receive the discussion
message through an e-mail from EDAS. Discussion messages can always be viewed
on the EDAS page for the paper. For details on how to initiate and post
discussion messages, please refer to the detailed instructions above.
- Discussion messages are
sent anonymously to all the TPC members who have been assigned as reviewers
for the paper as well as any delegated reviewers. You should not identify
yourself when you post discussion messages on EDAS. Reviewers may identify
each other as "Reviewer A", "Reviewer B" etc.
- The TPC lead of a paper
will have to check if the numerical scores are well justified by the written
review (and also catch short/low quality reviews), and follow up with
reviewers using EDAS requesting them to update their reviews.
- Also, please just do not
mark papers for "discuss" without adequate on-line discussion. During the TPC
meeting, the group members spend an average of 15 minute per paper and do not
have the time to *review* the paper in detail - they will depend quite a bit
upon the paper comments and on-line discussions. Kindly provide them with more
clarifications about why you have unresolved differences, and reasons for
accept/reject.
- If the paper truly needs
more attention and an extra authoritative review, kindly mark it for an extra
review. This year, we will have Area TPC chairs who
will find such a review for papers and help in decision making.
- The TPC-lead summary
(meta-review) is due by Oct. 17. An extension of a few days can only be
granted by the ATPC Chair who is responsible for the paper.
- The ATPC chair can view
the discussion messages and participate in the online discussions (if
necessary). ATPC Chairs will also be able to view reviewer identities.
Role of Area TPC
Chairs
For INFOCOM 2012, Area
TPC Chairs will play a critical role in the review process after the online
discussion phase is completed and the Lead TPC review/summary has been
submitted. The role of the Area TPC Chairs is to categorize the paper as Accept,
Discuss at TPC meeting or Reject. The Area TPC chairs may seek or provide
additional reviews. The Area TPC chairs will also lead the discussion groups at
the INFOCOM 2012 TPC meeting described below. Additional information about the
role of the Area TPC Chairs can be found here.
TPC
Meeting
The INFOCOM 2012 TPC
Meeting is scheduled on November 12, 2011 in
Details are on the INFOCOM 2012 website
at TPC Meeting Info
page. Please let us know if you will attend the meeting and the dinner by
filling out the TPC meeting page on
EDAS. All TPC members are invited to the TPC meeting. In order to put
together a high-quality program, it is important to have you participate in this
meeting. This is an invaluable experience to see how papers are actually accepted/rejected and which reviews matter and which reviews
do not. It is a way to tap into the decision mindset of your peers in the
community (especially for newer TPC members) so that you can write more
competitive papers to INFOCOM and other conferences.
We expect to have a more focused set of
papers at the TPC meeting with a higher acceptance probability (almost 40-50%)
compared to the overall conference acceptance rate of 20%. This will hopefully
bias the discussions to be positive and discuss papers which have good feedback
from the TPC review, TPC discussion and Area TPC processes.
We expect to have groups discussing
about 25--30 papers each, which will give an average of 15 min per paper (Note:
these numbers are tentative at this stage and we will know better by October
2011). This is why the on-line TPC discussion should not "punt" the hard work to
the TPC meeting and instead should give good feedback. Also, reviews having high
ratings with short reviews may be discounted. Area TPC chairs may choose to
bring in papers with lower numerical scores, but have substantive feedback
(possibly through an extra review).
Registering for TPC Meeting -
HOWTO
- How do I register for
the TPC meeting on EDAS?
You may visit
this page on EDAS. Alternatively,
when you log in to your EDAS page (http://edas.info), please click on the "My
TPCs" tab, you will then be shown a list of all the
conferences for which you are a TPC member. In the row for "INFOCOM 2012", you
will see the icon
under
the column "TPC meeting". Please click on the icon. Then answer the following
questions:
- (a) I will attend the
TPC meeting
- (b) I will attend
dinner
- (c) I will be staying
at the hotel (TBD)
- (d) I need vegetarian
meal at dinner
You may indicate in the
Comments field additional information such as "I will be arriving late", "I will
have to leave early" etc. You may indicate your dietary restrictions by updating
your profile on EDAS. There is a link provided to your profile in your TPC
meeting page. Please click on the "Submit" button when you are finished.
- Where can I find more
information about the TPC meeting location? Detailed information about the TPC meeting can
be found at the TPC
Meeting Info page on the INFOCOM 2012 website. Attendees are strongly
encouraged to confirm your travel and hotel bookings early to obtain good
rates in the
- How do I know if I
have already registered for the TPC meeting on EDAS?
You may visit
this page on EDAS. Alternatively,
when you log in to your EDAS page (http://edas.info), please click on the "My
TPCs" tab), you will then be shown a list of all the
conferences for which you are a TPC member. In the row for "INFOCOM 2012", you
will see the iconunder
the column "TPC meeting". Please click on the icon and you will see your earlier
response (both the radio buttons and the text box).
- I need a formal
registration/invitation letter to authorize my travel/reimbursement/Visa.
Where can I obtain this? Unfortunately, EDAS does not generate a
confirmation email or registration letter to attendees. If you need written
proof of registration/invitation for the TPC meeting, please contact Ms.
Debora Kingston (email) at IEEE
Communications Society.